TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks.

Spiking neural networks (SNNs) are attracting widespread interest due to their biological plausibility, energy efficiency, and powerful spatiotemporal information representation ability. Given the critical role of attention mechanisms in enhancing neural network performance, the integration of SNNs and attention mechanisms exhibits tremendous potential to deliver energy-efficient and high-performance computing paradigms. In this article, we present a novel temporal-channel joint attention mechanism for SNNs, referred to as TCJA-SNN. The proposed TCJA-SNN framework can effectively assess the significance of spike sequence from both spatial and temporal dimensions. More specifically, our essential technical contribution lies on: 1) we employ the squeeze operation to compress the spike stream into an average matrix. Then, we leverage two local attention mechanisms based on efficient 1-D convolutions to facilitate comprehensive feature extraction at the temporal and channel levels independently and 2) we introduce the cross-convolutional fusion (CCF) layer as a novel approach to model the interdependencies between the temporal and channel scopes. This layer effectively breaks the independence of these two dimensions and enables the interaction between features. Experimental results demonstrate that the proposed TCJA-SNN outperforms the state-of-the-art (SOTA) on all standard static and neuromorphic datasets, including Fashion-MNIST, CIFAR10, CIFAR100, CIFAR10-DVS, N-Caltech 101, and DVS128 Gesture. Furthermore, we effectively apply the TCJA-SNN framework to image generation tasks by leveraging a variation autoencoder. To the best of our knowledge, this study is the first instance where the SNN-attention mechanism has been employed for high-level classification and low-level generation tasks. Our implementation codes are available at https://github.com/ridgerchu/TCJA.

Interfacial Engineering of In2Se3/h-BN/CsPb(Br/I)3 Heterostructure Photodetector and Its Application in Automatic Obstacle Avoidance System.

Driven by the rapid development of autonomous vehicles, ultrasensitive photodetectors with high signal-to-noise ratio and ultraweak light detection capability are urgently needed. Due to its intriguing attributes, the emerging van der Waals material, indium selenide (In2Se3), has attracted extensive attention as an ultrasensitive photoactive material. However, the lack of an effective photoconductive gain mechanism in individual In2Se3 inhibits its further application. Herein, we propose a heterostructure photodetector consisting of an In2Se3 photoactive channel, a hexagonal boron nitride (h-BN) passivation layer, and a CsPb(Br/I)3 quantum dot gain layer. This device manifests a signal-to-noise ratio of 2 × 106 with responsivity of 2994 A/W and detectivity of 4.3 × 1014 Jones. Especially, it enables the detection of weak light as low as 0.03 µW/cm2. These performance characteristics are ascribed to the interfacial engineering. In2Se3 and CsPb(Br/I)3 with type-II band alignment promote the separation of photocarriers, while h-BN passivates the impurities on CsPb(Br/I)3 and promises a high-quality carrier transport interface. Furthermore, this device is successfully integrated into an automatic obstacle avoidance system, demonstrating promising application prospects in autonomous vehicles.

Local Manifold-Based Sparse Discriminant Learning for Feature Extraction of Hyperspectral Image.

Hyperspectral image (HSI) generally contains a complex manifold structure and strong sparse correlation in its nonlinear high-dimensional data space. However, the existing manifold learning and sparse learning methods usually consider the manifold structure and sparse relationship separately rather than combining manifold and sparse properties to discover the intrinsic information in the original data. To simultaneously reveal the complex sparse relation and manifold structure of HSI, a novel feature extraction (FE) method, called local manifold-based sparse discriminant learning (LMSDL), has been proposed on the basis of manifold learning and sparse representation (SR). The LMSDL method first designs a new sparse optimization model called local manifold-based SR (LMSR) to reveal the local manifold-based sparse structure of data. Then, two geometrical sparse graphs are constructed to represent the discriminant relationship between samples and the geometrical and sparse neighbors. An objective function is constructed via geometrical sparse graphs and reconstruction points to learn a projection matrix for FE. The LMSDL effectively reveals the complex sparse relation and manifold structure in high-dimensional data, and it enhances the representation ability of extracted features for HSI classification significantly. The experimental results on the three real HSI datasets show that the proposed LMSDL algorithm possesses better performance in comparison with some state-of-the-art FE methods.

DLPNet: A deep manifold network for feature extraction of hyperspectral imagery.

Deep learning has received increasing attention in recent years and it has been successfully applied for feature extraction (FE) of hyperspectral images. However, most deep learning methods fail to explore the manifold structure in hyperspectral image (HSI). To tackle this issue, a novel graph-based deep learning model, termed deep locality preserving neural network (DLPNet), was proposed in this paper. Traditional deep learning methods use random initialization to initialize network parameters. Different from that, DLPNet initializes each layer of the network by exploring the manifold structure in hyperspectral data. In the stage of network optimization, it designed a deep-manifold learning joint loss function to exploit graph embedding process while measuring the difference between the predictive value and the actual value, then the proposed model can take into account the extraction of deep features and explore the manifold structure of data simultaneously. Experimental results on real-world HSI datasets indicate that the proposed DLPNet performs significantly better than some state-of-the-art methods.

Dimensionality Reduction of Hyperspectral Imagery Based on Spatial-Spectral Manifold Learning.

The graph embedding (GE) methods have been widely applied for dimensionality reduction of hyperspectral imagery (HSI). However, a major challenge of GE is how to choose the proper neighbors for graph construction and explore the spatial information of HSI data. In this paper, we proposed an unsupervised dimensionality reduction algorithm called spatial-spectral manifold reconstruction preserving embedding (SSMRPE) for HSI classification. At first, a weighted mean filter (WMF) is employed to preprocess the image, which aims to reduce the influence of background noise. According to the spatial consistency property of HSI, SSMRPE utilizes a new spatial-spectral combined distance (SSCD) to fuse the spatial structure and spectral information for selecting effective spatial-spectral neighbors of HSI pixels. Then, it explores the spatial relationship between each point and its neighbors to adjust the reconstruction weights to improve the efficiency of manifold reconstruction. As a result, the proposed method can extract the discriminant features and subsequently improve the classification performance of HSI. The experimental results on the PaviaU and Salinas hyperspectral data sets indicate that SSMRPE can achieve better classification results in comparison with some state-of-the-art methods.